Probing and Fault Injection of Distributed Protocol Implementations

Ensuring that a distributed system with strict dependability constraints meets its prescribed speciication is a growing challenge that confronts software developers and system engineers. This paper presents a technique for probing and fault injection of fault-tolerant distributed protocols. The proposed technique, called script-driven probing and fault injection, can be used for studying the behavior of distributed systems and for detecting design and implementation errors of fault-tolerant protocols. The focus of this work is on fault injection techniques that can be used to demonstrate three aspects of a target protocol: i) detection of design or implementation errors, ii) identiication of violations of protocol speciications, and iii) insight into design decisions made by the implementors. The emphasis of our approach is on experimental techniques intended to identify speciic \problems" in a protocol or its implementation rather than the evaluation of system dependability through statistical metrics such as fault coverage. To demonstrate the capabilities of this technique, the paper describes a probing and fault injection tool, called the PFI tool (Probe/Fault Injection Tool), and a summary of several extensive experiments that studied the behavior of two protocols: the Transmission Control Protocol (TCP) 5,28] and the Group Membership Protocol (GMP) 20].